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VIDEO DISPLAY SYSTEM WITH TWO CONTROLLERS EACH ABLE TO 
SCALE AND BLEND RGB AND YUV SURFACES 

' ^This application is related to a^commonly assigned co-pending application filed 
5 simultaneously herewith pocket number 10442-6US) entitled "MULTIPLE 
DISPLAY DEVICE DIS/LAY CONTROLLER WITH VIDEO OVERLAY AND 
FULL SCREEN Vlpio OUTPUTS", the specification of which is hereby 
incorporated by reference. 

10 Field of the invention 

The present invention relates to a single controller system for multiple display 
output. 

Background of fhe Invention 

1 5 Video display controllers are known which are able to work with both RGB format 
video and YUV format video to combine two video surfaces from graphics 
memory to provide an overlay output. This is useful, for example, in the situation 
that a movie source in YUV format is obtained by decoding a video file from say 
a CD-ROM or DVD device, and written to a video surface in graphics memory. At 
the same time a computer environment desktop is stored in RGB format in 
another surface within graphics memory. These two surfaces can be combined to 
provide the YUV source within a window on the desktop. Because the video 
display controller handles the conversion, scaling and mixing of the two sources, 
the computer operating system or application program does not directly create 
25 the desired display, but rather instructs the display controller system to scale and 
combine the two sources as desired. The quality of display and efficiency of the 
computer system is thereby greatly enhanced. An example of such a system is 
described in US patent 5,764,201 to Ranganathan. 

In such displays controller systems, only one combined output of mixed format 
video is possible in which case the second output can only mirror what is being 
displayed by the first output Furthermore, since only two surfaces (one RGB and 
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one YUV) are being fetched one for each of only one RGB pixel path and only 
one YUV pixel path, the degree of flexibility is further limited, and it would be 
desirable to have greater control over surface selection, combination, scaling and 
output. 



, ^ Summary of the Invention 

/» / it isWierefore a first object of the present invention to provide an apparatus and a 
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methfcxf for controlling a single graphic controller system with a single memory 
iYZtlhZ'JJ. l!« v .!'jy *i ! ea ?i wo qispiay outputs, in which control of surface 

selectioV conversion, combination, scaling and output is afforded without the 

limitations\known in the prior art. 

ccording to the invention, there is provided a method l\ providing a display 
output for at least two display devices using a single/controller system. The 
method comprises: 

providing a first display controller able to rea/from a graphics memory at 
least two first surfaces into at least two first pixel/aths, convert the pixel format 
of at least one of the at least two first surfaces/scale at least one of the at least 
two first surfaces, and combine (blend and/br overlay) the at least two first 
surfaces, where at least one of the first pixe/paths supports any one of RGB and 
YUV pixel format and at least one of the cfocef paths supports at least RGB pixel 
format; providing a second display controller able to read from a graphics 
memory at least two second surfaces/convert the pixel format of at least one of 
the at least two second surfaces, scale at least one of the at least two second 
surfaces, and combine the at lea/ two second surfaces, where at least one of 
the pixel paths supports any one4f RGB and YUV pixel formats and at least one 
of the pixel paths supports at least an RGB pixel format; 

causing the first display controller to select and read the first surfaces, 
convert the first surfaces ir/o a like first format at least when the first surfaces are 
not all in the like first for/at. scaling at least one of the first surfaces, combining 
(blending and/or overfef/ing) the first surfaces to obtain a blended first surface, 
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and outputting the combined first surface to provide a fjjsfoutput stream of pixel 
data; 

causing the second display controller ttf select and read the second 
surfaces, convert the second surfaces into ajke second format at least when the 
5 second surfaces are not in the like second format, scaling at least one of the 
second surfaces, combining (blendirjgrand/or overlaying) the second surfaces to 
obtain a blended second surface^end outputting the combined second surface to 
provide a second output streafn of pixel data. In this way, flexibility is provided 
by selection of the first yd second surfaces as well as scaling and combining of 
10 the first and second ^surfaces, whether the surfaces are in RGB format, YUV 
format or mixed Rpfi/YUV format. It will be appreciated that any of the surfaces 
could be wmmoft between the two controllers. This enables the same surface to 
be displayed/fen the two displays in different ways, formats and blended with 
different sjmaces. 



While the preferred embodiment suggests that any of the scalers be part of the 
controllers, the functioning of any of these scalers can be carried out by using the 
scaling capabilities of the graphic controller system's drawing engine whereby 
that particular scaler may be removed from the preferred embodiment. 



Preferably, the first output stream and the second output stream are fed into a 
first multiplexer and a second multiplexer, an output of the first multiplexer being 
fed into a first display, and an output of the second multiplexer being fed into a 
second display, and the method further comprises causing the multiplexers to 
25 select a desired one of the output streams for display on the first and second 
displays. 



Also preferably, the first display controller reads two first surfaces, has at least 
one controllable color space converters outputting a selected one of RGB and 
30 YUV pixel format , and two scaling units scaling an output of the two color space 
converters, and a combining unit receiving an output of the two scaling units. The 
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method then preferably includes causing the two scaling units to scale each of 
the two first surfaces. While the preferred embodiment includes two scalers, 
fewer or more scalers can be used. As an example, the two scaling units can be 
replaced by one scaler scaling the output of the combining unit in a case where it 
5 is known that the two surfaces do not need to be scaled independently 

^single controller system may alternatively comprises a drawing engine scaler 
instead of any one or ail backend scalers responsive to a scaling command to 
pre-scate at least one surface in the graphics memory and output a scaled 
version flo a scaled surface in the graphics memory; and one of the steps of 
1 0 causing th^first display controller and causing the second display controller may 
comprise on^of scaling the at least one of the first surfaces and the at least one 
of the second Waces, respectively, using the drawing engine and reading the 
at least one of trte first surfaces and the at least one of the second surfaces, 
respectively, from the scaled surface. 
15 X 

While the description of the invention uses two controllers as the preferred 
embodiment, it can easily be extended or scaled to additional controllers. 

Two independent display controllers (primary display and secondary display 
20 controller) each able to generate a stream of pixel data and associated 
synchronization signals (syncs) from pixel data contained in a display memory 
(surfaces). Each display controller can combine (overlay or blend) two or more 
surfaces using two or more pixel paths with or without scaling in a wide variety of 
pixel formats and color spaces. (Note that the number of surfaces can be more 
than two while still remaining within the scope and spirit of the invention). This 
provides a great deal of capability and flexibility, which enables a host of useful 
new features to the end user. The two display controllers can drive a variety of 
output ports including any combination of RGB D/A converters, video encoder, 
and TDMS Panel Link or LCD interface. This permits many display combinations 
such as two RGB monitors, two TV monitors, two flat panel displays or any mix 
of them. 
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The primary display controller can read two surfaces from display memory into 
two pixel paths. Each surface can be in any one of a variety of pixel data formats 
such as YUV (4:2:0. 4:2:2 etc.) and RGB (8. 16, 24. 32). The primary display 
5 controller can combine (overlay or blend) the two surfaces together. One or both 
of the surfaces can optionally be re-scaled using either one of or all of the 
backend scalers before or after being combined using the appropriate backend 
scaler or the 3D drawing engine into a display surface in memory before even 
being read by the display controller. A fixed or programmable color space 
1 0 converter is included in one or both pixel data paths, so that combined pixels are 
in the same color space. For example, a surface can be in YCbCr or YUV format 
(video) and is converted in RGB (graphic) format before being combined with the 
RGB Windows desktop. Typically, but not exclusively, this is used to display a 
| video surface (video in a window) under a graphic overlay surface (e.g. 2D 

1 1 5 desktop). One of the overlay surface can be a graphic bitmap. Typically, but not 
S exclusively, this is used to display a DVD sub-picture overlay (or blended) 

= surface over a DVD decoded picture. 

3 The secondary display controller can also simultaneously read 2 surfaces from 

j 20 display memory into two pixel paths and combine them together. The secondary 
n disp,a y ^"troller can be identical to the primary display controller, or it can 

3 support a subset of the full feature set . For example, the secondary display 

controller's main surface could either be in YCbCr (4:4:4, 4:2:2. 4:1:1, 4:2:0) or 
RGBalpha (8, 16, 24, 32 bpp) format which allows for various types of surfaces to 
25 be displayed (grabbed video from video input port, S/W decoded video, graphic 
images etc.). The second overlay surface can be a graphic bitmap or other type 
of surface. Typically, but not exclusively, Ihis is used to display a DVD sub- 
picture overlay surface over a DVD decoded picture. Note that in single chip 
multi-display controllers of prior art, blending and scaling units are limited to one 
30 controller only and/or a total of only two pixel paths are present . This invention 
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requires the incorporation of at least one of such units and at least two pixel 
paths in each of at least two controllers in order to increase flexibility. 

One or more CRTCs can be set to convert a 16 or 32 bpp RGBalpha format into 
a YUV 4:2:2 ♦ alpha format. When RGB32 is output as YUV 4:2:2 ♦ alpha data 
(upper 8 bits out of RGB32), the alpha data is useful as a key signal. This mode 
is useful for outputting a Windows desktop surface as a standard video signal. It 
is also useful for using the video controller as a source of graphics plus key for 
professional video graphics and titling applications. 
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One of the main applications of the invention is where the second display can 
show a video image full screen while the main display maintains a separate 
image such as a fully interactive Windows desktop. The video from the second 
display can also be replicated on the first display within a window. For the video 
15 on both displays to have sub~picture blending, both controllers must have sub- 
picture blending units. 

Brief Descript ion of th« Drawing 

The invention will be better understood by way of the following detailed 
description of a preferred embodiment with reference to the appended drawings, 
in which: 

/|Jjj^> Fig. 1 is a schematic blocl/ diagram of a single controller system for 
providing dual display output acting to the preferred embodiment; and 
^ kO*^^ F ' 9 ' 2 ' s a scnemat M>lock diagram of a controller reading two surfaces of 
hl&T any of a plurality of vid?6 formats, carrying out conversion, if necessary, and 
f scaling and combing (overlaying and/or blending) in accordance with the 

preferred embodiment; and 
L^jk^KQ- 3 is a pictoral diagrarm of two different surfaces in graphics memory 
showing the result of combining with sub-picture blending and overlaying 

Detailed Description of th* Preferred Embotf m«n» 
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'"fit? Y Sh ° WS 3 h ' gh ,eve ' b,ock diagram of the invention - Tvvo CRTC 

controlWs (11) and (12) are capable of fetching 2 surfaces from memory (50) 
which caW SGRAM, SDRAM, or any other type of Random Access Memory. 
These surface can be in a variety of pixel formats including but not limited to 
5 RGB (8, 16.W 32) and YUV (4:2:0 , 4, 2, 2). The output of each CRTC can be 
displayed on a\CRT monitor, TV or flat Panel displays via appropriate converters, 
encoders and transmitters. The Multiplexers (33) and (34) allow routing of the 
output of each cWc to either display. This allows either display to receive the 
output from either GRTC. 
10 * 

' 5^7 Each \ RTC stains one ° r more backend Scaler (14) (refer to Figure 2) that 
allows the input surfaces to be re-scaled before being combined (overlaid, 
blended eV) with the other surface. Alternatively, prescaling of any surface could 
be done by\he drawing engine . A fixed or programmable color space converter 
(1 5) is included in one or both pixel data paths, so that combined pixels are in the 
same color spate. For example, a surface can be in YCbCr format (video) and is 
converted in RGB (graphic) format before being combined with the RGB 
Windows desktop, \ypically, but not exclusively, this is used to display a video 
surface (video in a Window) under a graphic overlay surface (e.g. 2D desktop). 
Each CRTC also contains a combining unit (or sub picture blending unit) for 
combining (overlaying orb|ending) the two surface. 

^/TypicallyV the graphics memory 50 and one or all encoders are external of the 
discrete device (ASIC) containing the CRTCs, the drawing engine and the 
25 graphics subsystem manager. 

Although conversion to RGB format is illustrated and preferred over conversion 
to YUV format, both types of conversion are possible. 

30 Figure 3 shows the results of combining a video surface with another surface (in 
this case a bitmap) using sub-picture blending (top right) or sub picture overlay 
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(bottom right). One application of this would be adding logos, subtitles etc. Note 
in the case of DVDs (as an example) where the language of the subtitles is user 
selectable, the sub-mtes are not embedded in the source of the YUV surface but 
are overlaid in realtime from a user selectable source. 

5 

e^pperation and design of the blending units and other units within the CRTCs 
(ScalerU and CSC 15) are general and well known to someone skilled in the art 
and are therefore not described here. 

'l^?lnsteadNof using the backend scaler, the 3D drawing engine 60 of the graphics 
controller \an also be used for pre-scaling surfaces and storing them in memory. 
While the scaling capabilities of the 3D drawing engine are typically used for 
scaling textured for 3D objects, the capability can be leveraged to scale any type 
of surface by treating the surface as a texture. This also allows a CRTC having 

15 only one scaler toNblend surfaces both of which need to be scaled, and this 
allows a CRTC havifto no scaler to blend surfaces in which at least one of the 
surfaces has been scaled prior to blending. 

/$/lt will be appreciated that CRTC1 and CRTC2 each read two surfaces from one 
20 memory 5o\and that one or both of these surfaces may be the same surfaces in 
which case tW same surface can be displayed in different ways. The present 
invention is als\> not limited to reading only two surfaces into two pixel paths, but 
a CRTC may be\designed to be controlled to read three or more surfaces into 
three or more pixe\ paths. 
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